A Study on Prosody and Discourse Structure in Cooperative Dialogues Short title: Prosody and Discourse Structure

This paper describes how well prosodic information correlates with the topic structure of a cooperative dialogue. To investigate this correlation systematically, rst we introduce the notion of utterance unit (UU) as a basic unit in conversations. We de ne the utterance unit by employing four principles. The Grammatical Principle is a syntactical criterion and the UU boundary is set wherever the period can be placed. The Pragmatic Principle says that each UU corresponds to a basic speech act. In other words, if two neighboring phrases correspond to di erent speech acts (for instance, acknowledgement and request), they should be taken as two di erent UUs. The Conversational Principle addresses the turn-taking aspect of conversations. An UU boundary should be placed wherever the speaker changes. Finally, the Prosodic Principle says that whenever a medium length or longer pause (750msec) is inserted between two phrases, they are to be taken as two di erent UUs. We apply these principles to a speech database containing about one and half hours of collected dialogue to split the dialogues into a sequence of UUs. We then classify the inter-UU boundaries based on the relationship between two neighboring UUs into four semantic categories: Topic Shift, Topic Continuation, Elaboration (or Clari cation), and Speech-Act Continuation. The prosodic parameters measured at each boundary are the onset fundamental frequency(F0), the nal F0, and the F0 maximal peak declination ratio{ the ratio of the current UU's maximal peak to that of the preceding UU. Our study shows how these prosodic parameters vary depending on the topic structure. Our results can be summarized as follows. 1) The onset F0 value tends to be higher when the topic is changed at the UU boundary. 2) The nal F0 value indicates nality and is much higher (on average) at speech-act continuation boundaries than those at other boundaries. 3) The maximal peak declination ratio re ects the degree of subordination to the preceding UU. That is, this ratio is lowest at elaboration boundaries and highest at topic shift boundaries. Finally, we discuss discourse structure identi cation via the prosodic parameters.